{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h5atBs3EyIsj"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "text_title:migration,automl,icn"
      },
      "source": [
        "# Vertex SDK: Train & deploy a TensorFlow model with hosted runtimes (aka pre-built containers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JE-aKjayJjkF"
      },
      "source": [
        "## Installation\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Iddo-8HmJjkH"
      },
      "source": [
        "Install the Google *cloud-storage* library as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xHvi6EFSJjkH"
      },
      "outputs": [],
      "source": [
        "! pip3 install google-cloud-storage"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ktRTgB8DJjkI"
      },
      "source": [
        "### Restart the Kernel\n",
        "\n",
        "Once you've installed the Vertex SDK and Google *cloud-storage*, you need to restart the notebook kernel so it can find the packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "x1g4mBdlJjkI"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "if not os.getenv(\"AUTORUN\"):\n",
        "    # Automatically restart kernel after installs\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin"
      },
      "source": [
        "## Before you begin\n",
        "\n",
        "### GPU run-time\n",
        "\n",
        "*Make sure you're running this notebook in a GPU runtime if you have that option. In Colab, select* **Runtime > Change Runtime Type > GPU**\n",
        "\n",
        "### Set up your GCP project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "1. [Select or create a GCP project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
        "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n",
        "\n",
        "3. [Enable the AutoML APIs and Compute Engine APIs.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component)\n",
        "\n",
        "4. [Google Cloud SDK](https://cloud.google.com/sdk) is already installed in AutoML Notebooks.\n",
        "\n",
        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
        "\n",
        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "autoset_project_id"
      },
      "outputs": [],
      "source": [
        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
        "    # Get your GCP project id from gcloud\n",
        "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
        "    PROJECT_ID = shell_output[0]\n",
        "    print(\"Project ID:\", PROJECT_ID)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_gcloud_project_id"
      },
      "outputs": [],
      "source": [
        "! gcloud config set project $PROJECT_ID"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jT0fijejJjkK"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable, which is used for operations\n",
        "throughout the rest of this notebook.  Below are regions supported forAutoML. We recommend when possible, to choose the region closest to you. \n",
        "\n",
        "Currently project resources must be in the `us-central1` region to use this API."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CTBlncfrJjkK"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J_croPBoJjkK"
      },
      "source": [
        "#### Timestamp\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append onto the name of resources which will be created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KKxStk5bJjkL"
      },
      "outputs": [],
      "source": [
        "from datetime import datetime\n",
        "\n",
        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcp_authenticate"
      },
      "source": [
        "### Authenticate your GCP account\n",
        "\n",
        "**If you are using AutoML Notebooks**, your environment is already\n",
        "authenticated. Skip this step.\n",
        "\n",
        "*Note: If you are on an AutoML notebook and run the cell, the cell knows to skip executing the authentication steps.*\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6aUuFIftxXiN"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import sys\n",
        "\n",
        "# If you are running this notebook in Colab, run this cell and follow the\n",
        "# instructions to authenticate your Google Cloud account. This provides access\n",
        "# to your Cloud Storage bucket and lets you submit training jobs and prediction\n",
        "# requests.\n",
        "\n",
        "# If on AutoML, then don't execute this code\n",
        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\"):\n",
        "    if \"google.colab\" in sys.modules:\n",
        "        from google.colab import auth as google_auth\n",
        "\n",
        "        google_auth.authenticate_user()\n",
        "\n",
        "    # If you are running this tutorial in a notebook locally, replace the string\n",
        "    # below with the path to your service account key and run this cell to\n",
        "    # authenticate your Google Cloud account.\n",
        "    else:\n",
        "        %env GOOGLE_APPLICATION_CREDENTIALS your_path_to_credentials.json\n",
        "\n",
        "    # Log in to your account on Google Cloud\n",
        "    ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDUAZaN3JjkL"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "This tutorial is designed to use training data that is in a public Cloud Storage bucket and a local Cloud Storage bucket for your batch predictions. You may alternatively use your own training data that you have stored in a local Cloud Storage bucket.\n",
        "\n",
        "Set the name of your Cloud Storage bucket below. It must be unique across all Cloud Storage buckets. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_NAME = \"[your-bucket-name]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "autoset_bucket"
      },
      "outputs": [],
      "source": [
        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
        "    BUCKET_NAME = PROJECT_ID + \"aip-\" + TIMESTAMP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yU6VuylGJjkM"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QVRixM4oJjkM"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION gs://$BUCKET_NAME"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mi-Pdh6QJjkN"
      },
      "source": [
        "Finally, validate access to your Cloud Storage bucket by examining its contents:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aqDptKpYJjkN"
      },
      "outputs": [],
      "source": [
        "! gsutil ls -al gs://$BUCKET_NAME"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fR9geV9pJjkO"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6PsrZK5lyIsu"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "import time\n",
        "\n",
        "from google.protobuf.json_format import MessageToJson\n",
        "from google.protobuf.struct_pb2 import Value\n",
        "from googleapiclient import discovery, errors"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lbv411XjJjkP"
      },
      "source": [
        "#### AutoML constants\n",
        "\n",
        "Setup up the following constants for AutoML:\n",
        "\n",
        "- `PARENT`: The AutoM location root path for dataset, model and endpoint resources."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gx8Eos8PJjkP"
      },
      "outputs": [],
      "source": [
        "# AutoM location root path for your dataset, model and endpoint resources\n",
        "PARENT = \"projects/\" + PROJECT_ID"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t5qygGYfyIsv"
      },
      "source": [
        "## Clients\n",
        "\n",
        "We use the Google APIs Client Library for Python to call the Vertex Training and Prediction API without manually constructing HTTP requests."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FUDz4l7GyIsv"
      },
      "outputs": [],
      "source": [
        "cloudml = discovery.build(\"ml\", \"v1\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZX-ma7Xj_5rS"
      },
      "source": [
        "## Prepare a trainer script"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lwO1GXK0_5rS"
      },
      "source": [
        "### Package assembly"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AcJLu3w3yIsv"
      },
      "outputs": [],
      "source": [
        "! rm -rf cifar\n",
        "! mkdir cifar\n",
        "! touch cifar/README.md\n",
        "\n",
        "setup_cfg = \"[egg_info]\\n\\\n",
        "tag_build =\\n\\\n",
        "tag_date = 0\"\n",
        "! echo \"$setup_cfg\" > cifar/setup.cfg\n",
        "\n",
        "setup_py = \"import setuptools\\n\\\n",
        "# Requires TensorFlow Datasets\\n\\\n",
        "setuptools.setup(\\n\\\n",
        "    install_requires=[\\n\\\n",
        "        'tensorflow_datasets==1.3.0',\\n\\\n",
        "    ],\\n\\\n",
        "    packages=setuptools.find_packages())\"\n",
        "! echo \"$setup_py\" > cifar/setup.py\n",
        "\n",
        "pkg_info = \"Metadata-Version: 1.0\\n\\\n",
        "Name: Custom Training CIFAR-10\\n\\\n",
        "Version: 0.0.0\\n\\\n",
        "Summary: Demonstration training script\\n\\\n",
        "Home-page: www.google.com\\n\\\n",
        "Author: Google\\n\\\n",
        "Author-email: aferlitsch@google.com\\n\\\n",
        "License: Public\\n\\\n",
        "Description: Demo\\n\\\n",
        "Platform: Vertex AI\"\n",
        "! echo \"$pkg_info\" > cifar/PKG-INFO\n",
        "\n",
        "! mkdir cifar/trainer\n",
        "! touch cifar/trainer/__init__.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7agySmGx_5rS"
      },
      "source": [
        "### Task.py contents"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RRGD4U4ryIsw"
      },
      "outputs": [],
      "source": [
        "%%writefile cifar/trainer/task.py\n",
        "import tensorflow_datasets as tfds\n",
        "import tensorflow as tf\n",
        "from tensorflow.python.client import device_lib\n",
        "import argparse\n",
        "import os\n",
        "import sys\n",
        "\n",
        "tfds.disable_progress_bar()\n",
        "\n",
        "parser = argparse.ArgumentParser()\n",
        "parser.add_argument('--model-dir', dest='model_dir',\n",
        "                    default='/tmp/saved_model', type=str, help='Model dir.')\n",
        "parser.add_argument('--lr', dest='lr',\n",
        "                    default=0.01, type=float,\n",
        "                    help='Learning rate.')\n",
        "parser.add_argument('--epochs', dest='epochs',\n",
        "                    default=10, type=int,\n",
        "                    help='Number of epochs.')\n",
        "parser.add_argument('--steps', dest='steps',\n",
        "                    default=200, type=int,\n",
        "                    help='Number of steps per epoch.')\n",
        "parser.add_argument('--distribute', dest='distribute', type=str, default='single',\n",
        "                    help='distributed training strategy')\n",
        "args = parser.parse_args()\n",
        "\n",
        "print('Python Version = {}'.format(sys.version))\n",
        "print('TensorFlow Version = {}'.format(tf.__version__))\n",
        "print('TF_CONFIG = {}'.format(os.environ.get('TF_CONFIG', 'Not found')))\n",
        "print('DEVICES', device_lib.list_local_devices())\n",
        "\n",
        "if args.distribute == 'single':\n",
        "    if tf.test.is_gpu_available():\n",
        "        strategy = tf.distribute.OneDeviceStrategy(device=\"/gpu:0\")\n",
        "    else:\n",
        "        strategy = tf.distribute.OneDeviceStrategy(device=\"/cpu:0\")\n",
        "elif args.distribute == 'mirror':\n",
        "    strategy = tf.distribute.MirroredStrategy()\n",
        "elif args.distribute == 'multi':\n",
        "    strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy()\n",
        "\n",
        "print('num_replicas_in_sync = {}'.format(strategy.num_replicas_in_sync))\n",
        "\n",
        "BUFFER_SIZE = 10000\n",
        "BATCH_SIZE = 64\n",
        "\n",
        "def make_datasets_unbatched():\n",
        "  def scale(image, label):\n",
        "    image = tf.cast(image, tf.float32)\n",
        "    image /= 255.0\n",
        "    return image, label\n",
        "\n",
        "  datasets, info = tfds.load(name='cifar10',\n",
        "                            with_info=True,\n",
        "                            as_supervised=True)\n",
        "  return datasets['train'].map(scale).cache().shuffle(BUFFER_SIZE).repeat()\n",
        "\n",
        "def build_and_compile_cnn_model():\n",
        "  model = tf.keras.Sequential([\n",
        "      tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(32, 32, 3)),\n",
        "      tf.keras.layers.MaxPooling2D(),\n",
        "      tf.keras.layers.Conv2D(32, 3, activation='relu'),\n",
        "      tf.keras.layers.MaxPooling2D(),\n",
        "      tf.keras.layers.Flatten(),\n",
        "      tf.keras.layers.Dense(10, activation='softmax')\n",
        "  ])\n",
        "  model.compile(\n",
        "      loss=tf.keras.losses.sparse_categorical_crossentropy,\n",
        "      optimizer=tf.keras.optimizers.SGD(learning_rate=args.lr),\n",
        "      metrics=['accuracy'])\n",
        "  return model\n",
        "\n",
        "NUM_WORKERS = strategy.num_replicas_in_sync\n",
        "GLOBAL_BATCH_SIZE = BATCH_SIZE * NUM_WORKERS\n",
        "train_dataset = make_datasets_unbatched().batch(GLOBAL_BATCH_SIZE)\n",
        "\n",
        "with strategy.scope():\n",
        "  model = build_and_compile_cnn_model()\n",
        "\n",
        "model.fit(x=train_dataset, epochs=args.epochs, steps_per_epoch=args.steps)\n",
        "model.save(args.model_dir)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7NYujrjG_5rT"
      },
      "source": [
        "### Store training script on your Cloud Storage bucket"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CA5_HWnEyIsz"
      },
      "outputs": [],
      "source": [
        "! rm -f cifar.tar cifar.tar.gz\n",
        "! tar cvf cifar.tar cifar\n",
        "! gzip cifar.tar\n",
        "! gsutil cp cifar.tar.gz gs://$BUCKET_NAME/trainer_cifar.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "text_create_and_deploy_model:migration"
      },
      "source": [
        "## Train a model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0oqIBOSnJjkW"
      },
      "source": [
        "### [projects.jobs.create](https://cloud.google.com/ai-platform/training/docs/reference/rest/v1/projects.jobs/create)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d1hFxaOKyIs2"
      },
      "source": [
        "#### Request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FmBnuvysyIs2"
      },
      "outputs": [],
      "source": [
        "JOB_NAME = \"custom_job_TF_\" + TIMESTAMP\n",
        "\n",
        "TRAINING_INPUTS = {\n",
        "    \"scaleTier\": \"CUSTOM\",\n",
        "    \"masterType\": \"n1-standard-4\",\n",
        "    \"masterConfig\": {\"acceleratorConfig\": {\"count\": \"1\", \"type\": \"NVIDIA_TESLA_K80\"}},\n",
        "    \"packageUris\": [\"gs://\" + BUCKET_NAME + \"/trainer_cifar.tar.gz\"],\n",
        "    \"pythonModule\": \"trainer.task\",\n",
        "    \"args\": [\n",
        "        \"--model-dir=\" + \"gs://{}/{}\".format(BUCKET_NAME, JOB_NAME),\n",
        "        \"--epochs=\" + str(20),\n",
        "        \"--steps=\" + str(100),\n",
        "        \"--distribute=\" + \"single\",\n",
        "    ],\n",
        "    \"region\": REGION,\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"pythonVersion\": \"3.7\",\n",
        "}\n",
        "\n",
        "body = {\"jobId\": JOB_NAME, \"trainingInput\": TRAINING_INPUTS}\n",
        "\n",
        "request = cloudml.projects().jobs().create(parent=PARENT)\n",
        "request.body = json.loads(json.dumps(body, indent=2))\n",
        "\n",
        "print(json.dumps(json.loads(request.to_json()), indent=2))\n",
        "\n",
        "request = cloudml.projects().jobs().create(parent=PARENT, body=body)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"uri\": \"https://ml.googleapis.com/v1/projects/migration-ucaip-training/jobs?alt=json\",\n",
        "  \"method\": \"POST\",\n",
        "  \"body\": {\n",
        "    \"jobId\": \"custom_job_TF_20210325211532\",\n",
        "    \"trainingInput\": {\n",
        "      \"scaleTier\": \"CUSTOM\",\n",
        "      \"masterType\": \"n1-standard-4\",\n",
        "      \"masterConfig\": {\n",
        "        \"acceleratorConfig\": {\n",
        "          \"count\": \"1\",\n",
        "          \"type\": \"NVIDIA_TESLA_K80\"\n",
        "        }\n",
        "      },\n",
        "      \"packageUris\": [\n",
        "        \"gs://migration-ucaip-trainingaip-20210325211532/trainer_cifar.tar.gz\"\n",
        "      ],\n",
        "      \"pythonModule\": \"trainer.task\",\n",
        "      \"args\": [\n",
        "        \"--model-dir=gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "        \"--epochs=20\",\n",
        "        \"--steps=100\",\n",
        "        \"--distribute=single\"\n",
        "      ],\n",
        "      \"region\": \"us-central1\",\n",
        "      \"runtimeVersion\": \"2.1\",\n",
        "      \"pythonVersion\": \"3.7\"\n",
        "    }\n",
        "  },\n",
        "  \"headers\": {\n",
        "    \"accept\": \"application/json\",\n",
        "    \"accept-encoding\": \"gzip, deflate\",\n",
        "    \"user-agent\": \"(gzip)\",\n",
        "    \"x-goog-api-client\": \"gdcl/1.12.8 gl-python/3.7.8\"\n",
        "  },\n",
        "  \"methodId\": \"ml.projects.jobs.create\",\n",
        "  \"resumable\": null,\n",
        "  \"response_callbacks\": [],\n",
        "  \"_in_error_state\": false,\n",
        "  \"body_size\": 0,\n",
        "  \"resumable_uri\": null,\n",
        "  \"resumable_progress\": 0\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w9N482wpyIs2"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cCL6ohxxyIs3"
      },
      "outputs": [],
      "source": [
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zV5RiSOmyIs3"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PFIKWKcNyIs3"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"jobId\": \"custom_job_TF_20210325211532\",\n",
        "  \"trainingInput\": {\n",
        "    \"scaleTier\": \"CUSTOM\",\n",
        "    \"masterType\": \"n1-standard-4\",\n",
        "    \"packageUris\": [\n",
        "      \"gs://migration-ucaip-trainingaip-20210325211532/trainer_cifar.tar.gz\"\n",
        "    ],\n",
        "    \"pythonModule\": \"trainer.task\",\n",
        "    \"args\": [\n",
        "      \"--model-dir=gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "      \"--epochs=20\",\n",
        "      \"--steps=100\",\n",
        "      \"--distribute=single\"\n",
        "    ],\n",
        "    \"region\": \"us-central1\",\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"pythonVersion\": \"3.7\",\n",
        "    \"masterConfig\": {\n",
        "      \"acceleratorConfig\": {\n",
        "        \"count\": \"1\",\n",
        "        \"type\": \"NVIDIA_TESLA_K80\"\n",
        "      }\n",
        "    }\n",
        "  },\n",
        "  \"createTime\": \"2021-03-25T21:15:40Z\",\n",
        "  \"state\": \"QUEUED\",\n",
        "  \"trainingOutput\": {},\n",
        "  \"etag\": \"dH4whflp8Fg=\"\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "training_pipeline_id:migration,new,response"
      },
      "outputs": [],
      "source": [
        "# The full unique ID for the custom training job\n",
        "custom_training_id = f'{PARENT}/jobs/{response[\"jobId\"]}'\n",
        "# The short numeric ID for the custom training job\n",
        "custom_training_short_id = response[\"jobId\"]\n",
        "\n",
        "print(custom_training_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pq4osl_GyIs3"
      },
      "source": [
        "### [projects.jobs.get](https://cloud.google.com/ai-platform/training/docs/reference/rest/v1/projects.jobs/get)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JIENR5rRyIs3"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DWPhOIMvyIs3"
      },
      "outputs": [],
      "source": [
        "request = cloudml.projects().jobs().get(name=custom_training_id)\n",
        "\n",
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P6-XV2JUyIs4"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pmm7CEQayIs4"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"jobId\": \"custom_job_TF_20210325211532\",\n",
        "  \"trainingInput\": {\n",
        "    \"scaleTier\": \"CUSTOM\",\n",
        "    \"masterType\": \"n1-standard-4\",\n",
        "    \"packageUris\": [\n",
        "      \"gs://migration-ucaip-trainingaip-20210325211532/trainer_cifar.tar.gz\"\n",
        "    ],\n",
        "    \"pythonModule\": \"trainer.task\",\n",
        "    \"args\": [\n",
        "      \"--model-dir=gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "      \"--epochs=20\",\n",
        "      \"--steps=100\",\n",
        "      \"--distribute=single\"\n",
        "    ],\n",
        "    \"region\": \"us-central1\",\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"pythonVersion\": \"3.7\",\n",
        "    \"masterConfig\": {\n",
        "      \"acceleratorConfig\": {\n",
        "        \"count\": \"1\",\n",
        "        \"type\": \"NVIDIA_TESLA_K80\"\n",
        "      }\n",
        "    }\n",
        "  },\n",
        "  \"createTime\": \"2021-03-25T21:15:40Z\",\n",
        "  \"state\": \"PREPARING\",\n",
        "  \"trainingOutput\": {},\n",
        "  \"etag\": \"eLnYfClHtKU=\"\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "trainingpipelines_get:migration,new,wait"
      },
      "outputs": [],
      "source": [
        "while True:\n",
        "    response = cloudml.projects().jobs().get(name=custom_training_id).execute()\n",
        "    if response[\"state\"] != \"SUCCEEDED\":\n",
        "        print(\"Training job has not completed:\", response[\"state\"])\n",
        "        if response[\"state\"] == \"FAILED\":\n",
        "            break\n",
        "    else:\n",
        "        break\n",
        "    time.sleep(20)\n",
        "\n",
        "# model artifact output directory on Google Cloud Storage\n",
        "model_artifact_dir = response[\"trainingInput\"][\"args\"][0].split(\"=\")[-1]\n",
        "print(\"artifact location  \" + model_artifact_dir)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQ5FRbQtyIs4"
      },
      "source": [
        "### Serving function for trained model (image data)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6GpykrtAyIs4"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "model = tf.keras.models.load_model(model_artifact_dir)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Hj6KWo31yIs4"
      },
      "outputs": [],
      "source": [
        "CONCRETE_INPUT = \"numpy_inputs\"\n",
        "\n",
        "\n",
        "def _preprocess(bytes_input):\n",
        "    decoded = tf.io.decode_jpeg(bytes_input, channels=3)\n",
        "    decoded = tf.image.convert_image_dtype(decoded, tf.float32)\n",
        "    resized = tf.image.resize(decoded, size=(32, 32))\n",
        "    rescale = tf.cast(resized / 255.0, tf.float32)\n",
        "    return rescale\n",
        "\n",
        "\n",
        "@tf.function(input_signature=[tf.TensorSpec([None], tf.string)])\n",
        "def preprocess_fn(bytes_inputs):\n",
        "    decoded_images = tf.map_fn(\n",
        "        _preprocess, bytes_inputs, dtype=tf.float32, back_prop=False\n",
        "    )\n",
        "    return {\n",
        "        CONCRETE_INPUT: decoded_images\n",
        "    }  # User needs to make sure the key matches model's input\n",
        "\n",
        "\n",
        "m_call = tf.function(model.call).get_concrete_function(\n",
        "    [tf.TensorSpec(shape=[None, 32, 32, 3], dtype=tf.float32, name=CONCRETE_INPUT)]\n",
        ")\n",
        "\n",
        "\n",
        "@tf.function(\n",
        "    input_signature=[tf.TensorSpec([None], tf.string), tf.TensorSpec([None], tf.string)]\n",
        ")\n",
        "def serving_fn(bytes_inputs, key):\n",
        "    images = preprocess_fn(bytes_inputs)\n",
        "    prob = m_call(**images)\n",
        "    return {\"prediction\": prob, \"key\": key}\n",
        "\n",
        "\n",
        "tf.saved_model.save(\n",
        "    model,\n",
        "    model_artifact_dir,\n",
        "    signatures={\n",
        "        \"serving_default\": serving_fn,\n",
        "    },\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wlu4ZnUKyIs5"
      },
      "outputs": [],
      "source": [
        "loaded = tf.saved_model.load(model_artifact_dir)\n",
        "\n",
        "tensors_specs = list(loaded.signatures[\"serving_default\"].structured_input_signature)\n",
        "print(\"Tensors specs:\", tensors_specs)\n",
        "\n",
        "input_name = [v for k, v in tensors_specs[1].items() if k != \"key\"][0].name\n",
        "print(\"Bytes input tensor name:\", input_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "Tensors specs: [(), {'bytes_inputs': TensorSpec(shape=(None,), dtype=tf.string, name='bytes_inputs'), 'key': TensorSpec(shape=(None,), dtype=tf.string, name='key')}]\n",
        "Bytes input tensor name: bytes_inputs\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a7H7jhulyIs7"
      },
      "source": [
        "## Make batch predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fEUvTTuTyIs7"
      },
      "source": [
        "### Prepare files for batch prediction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HOOHm0SpyIs7"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "import json\n",
        "\n",
        "import cv2\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "\n",
        "(_, _), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()\n",
        "\n",
        "test_image_1, test_label_1 = x_test[0], y_test[0]\n",
        "test_image_2, test_label_2 = x_test[1], y_test[1]\n",
        "\n",
        "cv2.imwrite(\"tmp1.jpg\", (test_image_1 * 255).astype(np.uint8))\n",
        "cv2.imwrite(\"tmp2.jpg\", (test_image_2 * 255).astype(np.uint8))\n",
        "\n",
        "gcs_input_uri = \"gs://\" + BUCKET_NAME + \"/\" + \"test.json\"\n",
        "with tf.io.gfile.GFile(gcs_input_uri, \"w\") as f:\n",
        "    for img in [\"tmp1.jpg\", \"tmp2.jpg\"]:\n",
        "        bytes = tf.io.read_file(img)\n",
        "        b64str = base64.b64encode(bytes.numpy()).decode(\"utf-8\")\n",
        "        f.write(json.dumps({\"key\": img, input_name: {\"b64\": b64str}}) + \"\\n\")\n",
        "\n",
        "! gsutil cat $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\"key\": \"tmp1.jpg\", \"bytes_inputs\": {\"b64\": \"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAgACADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6E1zw/qemaZY669mkdtqsTPZTpMH85Y3KMcKeOR36444NZGj2/ibWPHaaPeHSLXRbq3jSw1O7u3V3u9zb0ZAh+QIFO4EkliCBjnwv9lfxtrviTxBbW974le/0nQ/h5ohms7m4b92bhVlkEfPIDuwJ6gyADgCuWh1fxP8As6/tGad8H5PiRrHjW6tNd1O/iXUr5Z7mx0uSZlinHODiRQCqrgGTGPmwPyqfClGlnM6Em3TSi/N3Wtnto015H6y+MK08kp14QSqScle6tFxel0+6aZ9d6/rvhXwH4407wWtq+uSXth9pa5jcwKUBIbyxzkL0Ock8nHQV2x0NtN0Gw8a6PDOunXc3liO5GZIGxwG6YBxx1x0zkV4L8Xfij4k8X/Gr4V+HdJtDpdgui3GoajJBAXlkuGvNoUEDcD5MYyuN3zEnpX0B4Q+Iunafdap8OPFCG/sL+PzLkGNgbQB1O7Jxh1JOCOvHXNfUYrh/LPqMo0oKDgvdl10117nzGD4izR5hGdWcp8zs4+umisflx8DNXi/Z/wDHviPTfiP4g+x2WieFtV03U5r9miLw2ilonTIySWijZCB6Yr2X4R/tQT/tC/s56f8AGn4C/AvxTrXiq7jksW1G78NxRlNiRxIrzO5EwiVHAePAfeoO1lIrqv2pf2Xz+1t+z3feC9E1GLSvE2paQtraa1cISXiEqu9tKVydrbMZ5Kkg8jIr234a/Bq7+EngjQPAng3wzB/ZOl6ZFa2tpp/yeWiqFB2Hq2ASeuTz15r9ixHBa+vSp1JXpxXuy6vyfpbXuz8jocUyWCVSirTb1j09V95e+E3hnwXr8dn8QPjLaSWZBguP+EcudKSW6gnSMfLHOrcQh2djCSAxY5BxkzfEDx1H4n8ZyvpEC2WnMAwighMe8hvl3gZyQCB15K5xWNq3iKbVNVk8MW91NZzxLllkt9jL2z0I/DrXCeG47T4seNL3wN4c1nULKPTY2GoX8YYNcSkfKisxwis2ASMnk9AK7f8AiHuQ47CulWlKzfM7S5W+vRfgZQ47zvA4qNako3irK8eZLpfVn//Z\"}}\n",
        "{\"key\": \"tmp2.jpg\", \"bytes_inputs\": {\"b64\": \"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAgACADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD8V5UubRQlxlSvDAtyD6dadbW91fK8lrFI6o6KzrnALHCj8cH8jX3J+1V+wR8adOjsrDxR8EPhzohsg13qKfD+zddRWBF2u0sR42AnIzjJAAzzXmnwh/Yk+D3jX4Q6h478cftgaX4Al/tR4f8AhHdf0eRruVI+IpdkbFiWLsAqgnrXZLBVFWcI6/gc0MVSlSU2eZaX+zdr954Nv/EEt7FNeWyrJHZ2moRn93tYsTuwcg7OBz19q8sa7AUEMf8AvqvoHwX+yz8Vb74gXtn4M+Euq/EbSYpV+y6vf2txptrMOAz+XIysR0xu9M4qf9pn9mf4jJoNprJ+BGgeCn0mHZfQ2OqRl793fAZUDkkAbcd8k1pUw1OUE6e/bf8AEVOs1JqT3P19/aT/AOCMf7RH7Qfx5134zeNf2z7S18Q+PkSWWDSb6406BrSMFYrWNCCAsakDbnOSSeTXg+sf8G3viHwt49ez1jxdY6zqds1veTwT+MzBdqJWnWCYb0DhXe3n2sOGMD4J2HH7IfD3xnc/EPwl4Y8R6t458M28y+EL1NRh1nS3vGXV3a1+w3S4mjCwxxpdCaFSjTNLGRImwk+A6f8AAL9oH4gaX4+tf+Ckn7Vfw4+I2k3fiW6m+HOneFNPn0WDw9piTLLbuUiYGWZsCNYp/tMtqiSbL+b7RMrqvWxVDKamZ89BOg03Q9+deupOpBRotU1CM4OMak/aSUIxkouTbUjmllc0qic60XrGNldX/dtNr/n2+aS5r3XI3ytKz+Jof+CN2r6LYHU/ibqOo2iQzFmmn8eXLfugMbDhwMcdeprg/iV+zX+zx8O9Mu9f8NaRplw9oSr6g0sl0BgdBNMzZ+i9K+svi9P+yv8ADAnRfhl4MfxNdhSDe63fzS2sJHdYpHbfjtu/KvhL9ub4tarruhy2JvJMsdjJFGFj28gKqrgKo9B6VhlvEGMzfDxm8M6N+kpRlJeT5dE/mwoZDiMO+evVb8j/2Q==\"}}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7YXC_0bCyIs8"
      },
      "source": [
        "### [projects.jobs.create](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects.jobs/create)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pAGChwNCyIs8"
      },
      "source": [
        "#### Request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rbUvA_wByIs8"
      },
      "outputs": [],
      "source": [
        "body = {\n",
        "    \"jobId\": \"custom_job_TF_pred_\" + TIMESTAMP,\n",
        "    \"prediction_input\": {\n",
        "        \"input_paths\": gcs_input_uri,\n",
        "        \"output_path\": \"gs://\" + f\"{BUCKET_NAME}/batch_output/\",\n",
        "        \"data_format\": \"JSON\",\n",
        "        \"runtime_version\": \"2.1\",\n",
        "        \"uri\": model_artifact_dir,\n",
        "        \"region\": \"us-central1\",\n",
        "    },\n",
        "}\n",
        "\n",
        "request = (\n",
        "    cloudml.projects()\n",
        "    .jobs()\n",
        "    .create(\n",
        "        parent=PARENT,\n",
        "    )\n",
        ")\n",
        "request.body = json.loads(json.dumps(body, indent=2))\n",
        "\n",
        "print(json.dumps(json.loads(request.to_json()), indent=2))\n",
        "\n",
        "request = cloudml.projects().jobs().create(parent=PARENT, body=body)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"uri\": \"https://ml.googleapis.com/v1/projects/migration-ucaip-training/jobs?alt=json\",\n",
        "  \"method\": \"POST\",\n",
        "  \"body\": {\n",
        "    \"jobId\": \"custom_job_TF_pred_20210325211532\",\n",
        "    \"prediction_input\": {\n",
        "      \"input_paths\": \"gs://migration-ucaip-trainingaip-20210325211532/test.json\",\n",
        "      \"output_path\": \"gs://migration-ucaip-trainingaip-20210325211532/batch_output/\",\n",
        "      \"data_format\": \"JSON\",\n",
        "      \"runtime_version\": \"2.1\",\n",
        "      \"uri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "      \"region\": \"us-central1\"\n",
        "    }\n",
        "  },\n",
        "  \"headers\": {\n",
        "    \"accept\": \"application/json\",\n",
        "    \"accept-encoding\": \"gzip, deflate\",\n",
        "    \"user-agent\": \"(gzip)\",\n",
        "    \"x-goog-api-client\": \"gdcl/1.12.8 gl-python/3.7.8\"\n",
        "  },\n",
        "  \"methodId\": \"ml.projects.jobs.create\",\n",
        "  \"resumable\": null,\n",
        "  \"response_callbacks\": [],\n",
        "  \"_in_error_state\": false,\n",
        "  \"body_size\": 0,\n",
        "  \"resumable_uri\": null,\n",
        "  \"resumable_progress\": 0\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8yue_3fjyIs8"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zmiM9YC1yIs8"
      },
      "outputs": [],
      "source": [
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "itggY5N6yIs8"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gOstdZbnyIs9"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"jobId\": \"custom_job_TF_pred_20210325211532\",\n",
        "  \"predictionInput\": {\n",
        "    \"dataFormat\": \"JSON\",\n",
        "    \"inputPaths\": [\n",
        "      \"gs://migration-ucaip-trainingaip-20210325211532/test.json\"\n",
        "    ],\n",
        "    \"outputPath\": \"gs://migration-ucaip-trainingaip-20210325211532/batch_output/\",\n",
        "    \"region\": \"us-central1\",\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"uri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "    \"framework\": \"TENSORFLOW\"\n",
        "  },\n",
        "  \"createTime\": \"2021-03-25T21:34:56Z\",\n",
        "  \"state\": \"QUEUED\",\n",
        "  \"predictionOutput\": {\n",
        "    \"outputPath\": \"gs://migration-ucaip-trainingaip-20210325211532/batch_output/\"\n",
        "  },\n",
        "  \"etag\": \"QwNOFOfoKdY=\"\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "037bd38edf14"
      },
      "outputs": [],
      "source": [
        "# The full unique ID for the batch prediction job\n",
        "batch_job_id = PARENT + \"/jobs/\" + response[\"jobId\"]\n",
        "\n",
        "print(batch_job_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j_d5kInfyIs9"
      },
      "source": [
        "### [projects.jobs.get](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects.jobs/get)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lbnf9fXUyIs9"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GvQTYRA2yIs9"
      },
      "outputs": [],
      "source": [
        "request = cloudml.projects().jobs().get(name=batch_job_id)\n",
        "\n",
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSJvKfb5yIs9"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aOPpWjd8yIs9"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"jobId\": \"custom_job_TF_pred_20210325211532\",\n",
        "  \"predictionInput\": {\n",
        "    \"dataFormat\": \"JSON\",\n",
        "    \"inputPaths\": [\n",
        "      \"gs://migration-ucaip-trainingaip-20210325211532/test.json\"\n",
        "    ],\n",
        "    \"outputPath\": \"gs://migration-ucaip-trainingaip-20210325211532/batch_output/\",\n",
        "    \"region\": \"us-central1\",\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"uri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "    \"framework\": \"TENSORFLOW\"\n",
        "  },\n",
        "  \"createTime\": \"2021-03-25T21:34:56Z\",\n",
        "  \"state\": \"QUEUED\",\n",
        "  \"predictionOutput\": {\n",
        "    \"outputPath\": \"gs://migration-ucaip-trainingaip-20210325211532/batch_output/\"\n",
        "  },\n",
        "  \"etag\": \"NSbtn4XnbbU=\"\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batchpredictionjobs_get:migration,new,wait"
      },
      "outputs": [],
      "source": [
        "while True:\n",
        "    response = request = cloudml.projects().jobs().get(name=batch_job_id).execute()\n",
        "    if response[\"state\"] != \"SUCCEEDED\":\n",
        "        print(\"The job has not completed:\", response[\"state\"])\n",
        "        if response[\"state\"] == \"FAILED\":\n",
        "            break\n",
        "    else:\n",
        "        folder = response[\"predictionInput\"][\"outputPath\"][:-1]\n",
        "        ! gsutil ls $folder/prediction*\n",
        "\n",
        "        ! gsutil cat $folder/prediction*\n",
        "        break\n",
        "    time.sleep(60)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "gs://migration-ucaip-trainingaip-20210325211532/batch_output/prediction.errors_stats-00000-of-00001\n",
        "gs://migration-ucaip-trainingaip-20210325211532/batch_output/prediction.results-00000-of-00001\n",
        "{\"prediction\": [0.033321816474199295, 0.052459586411714554, 0.1548144668340683, 0.11401787400245667, 0.17382358014583588, 0.09015274047851562, 0.19865882396697998, 0.10446511209011078, 0.029874442145228386, 0.048411525785923004], \"key\": \"tmp1.jpg\"}\n",
        "{\"prediction\": [0.03346974775195122, 0.05255022272467613, 0.15449963510036469, 0.11388237029314041, 0.17408262193202972, 0.08989296853542328, 0.19814379513263702, 0.10520868003368378, 0.02989153563976288, 0.04837837815284729], \"key\": \"tmp2.jpg\"}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tvjk99efyIs9"
      },
      "source": [
        "## Make online predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gi_osqnJyIs4"
      },
      "source": [
        "### Deploy the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "guF8Wj1hyIs5"
      },
      "source": [
        "### [projects.models.create](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects.models/create)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g-I23pXRyIs5"
      },
      "source": [
        "#### Request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rjRGd04iyIs5"
      },
      "outputs": [],
      "source": [
        "request = cloudml.projects().models().create(parent=PARENT)\n",
        "request.body = json.loads(json.dumps({\"name\": \"custom_job_TF_\" + TIMESTAMP}, indent=2))\n",
        "\n",
        "print(json.dumps(json.loads(request.to_json()), indent=2))\n",
        "\n",
        "request = (\n",
        "    cloudml.projects()\n",
        "    .models()\n",
        "    .create(parent=PARENT, body={\"name\": \"custom_job_TF_\" + TIMESTAMP})\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"uri\": \"https://ml.googleapis.com/v1/projects/migration-ucaip-training/models?alt=json\",\n",
        "  \"method\": \"POST\",\n",
        "  \"body\": {\n",
        "    \"name\": \"custom_job_TF_20210325211532\"\n",
        "  },\n",
        "  \"headers\": {\n",
        "    \"accept\": \"application/json\",\n",
        "    \"accept-encoding\": \"gzip, deflate\",\n",
        "    \"user-agent\": \"(gzip)\",\n",
        "    \"x-goog-api-client\": \"gdcl/1.12.8 gl-python/3.7.8\"\n",
        "  },\n",
        "  \"methodId\": \"ml.projects.models.create\",\n",
        "  \"resumable\": null,\n",
        "  \"response_callbacks\": [],\n",
        "  \"_in_error_state\": false,\n",
        "  \"body_size\": 0,\n",
        "  \"resumable_uri\": null,\n",
        "  \"resumable_progress\": 0\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nAwWTbADyIs5"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j0-FL4T6yIs6"
      },
      "outputs": [],
      "source": [
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9sK2H5jZyIs6"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DU5a9qR4yIs6"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"name\": \"projects/migration-ucaip-training/models/custom_job_TF_20210325211532\",\n",
        "  \"regions\": [\n",
        "    \"us-central1\"\n",
        "  ],\n",
        "  \"etag\": \"fFH1QQbH3tA=\"\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cf37425915f9"
      },
      "outputs": [],
      "source": [
        "# The full unique ID for the training pipeline\n",
        "model_id = response[\"name\"]\n",
        "# The short numeric ID for the training pipeline\n",
        "model_short_name = model_id.split(\"/\")[-1]\n",
        "\n",
        "print(model_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vJrwNIxJyIs6"
      },
      "source": [
        "### [projects.models.versions.create](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects.models.versions/create)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SFIKWys2yIs6"
      },
      "source": [
        "#### Request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cD2V3bjhyIs6"
      },
      "outputs": [],
      "source": [
        "version = {\n",
        "    \"name\": \"custom_job_TF_\" + TIMESTAMP,\n",
        "    \"deploymentUri\": model_artifact_dir,\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"framework\": \"TENSORFLOW\",\n",
        "    \"pythonVersion\": \"3.7\",\n",
        "    \"machineType\": \"mls1-c1-m2\",\n",
        "}\n",
        "\n",
        "request = (\n",
        "    cloudml.projects()\n",
        "    .models()\n",
        "    .versions()\n",
        "    .create(\n",
        "        parent=model_id,\n",
        "    )\n",
        ")\n",
        "request.body = json.loads(json.dumps(version, indent=2))\n",
        "\n",
        "print(json.dumps(json.loads(request.to_json()), indent=2))\n",
        "\n",
        "request = cloudml.projects().models().versions().create(parent=model_id, body=version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"uri\": \"https://ml.googleapis.com/v1/projects/migration-ucaip-training/models/custom_job_TF_20210325211532/versions?alt=json\",\n",
        "  \"method\": \"POST\",\n",
        "  \"body\": {\n",
        "    \"name\": \"custom_job_TF_20210325211532\",\n",
        "    \"deploymentUri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "    \"runtimeVersion\": \"2.1\",\n",
        "    \"framework\": \"TENSORFLOW\",\n",
        "    \"pythonVersion\": \"3.7\",\n",
        "    \"machineType\": \"mls1-c1-m2\"\n",
        "  },\n",
        "  \"headers\": {\n",
        "    \"accept\": \"application/json\",\n",
        "    \"accept-encoding\": \"gzip, deflate\",\n",
        "    \"user-agent\": \"(gzip)\",\n",
        "    \"x-goog-api-client\": \"gdcl/1.12.8 gl-python/3.7.8\"\n",
        "  },\n",
        "  \"methodId\": \"ml.projects.models.versions.create\",\n",
        "  \"resumable\": null,\n",
        "  \"response_callbacks\": [],\n",
        "  \"_in_error_state\": false,\n",
        "  \"body_size\": 0,\n",
        "  \"resumable_uri\": null,\n",
        "  \"resumable_progress\": 0\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TGZSWzgQyIs6"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GS5at2kdyIs7"
      },
      "outputs": [],
      "source": [
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PPzFKPcayIs7"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ql_RUaDGyIs7"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"name\": \"projects/migration-ucaip-training/operations/create_custom_job_TF_20210325211532_custom_job_TF_20210325211532-1616708521927\",\n",
        "  \"metadata\": {\n",
        "    \"@type\": \"type.googleapis.com/google.cloud.ml.v1.OperationMetadata\",\n",
        "    \"createTime\": \"2021-03-25T21:42:02Z\",\n",
        "    \"operationType\": \"CREATE_VERSION\",\n",
        "    \"modelName\": \"projects/migration-ucaip-training/models/custom_job_TF_20210325211532\",\n",
        "    \"version\": {\n",
        "      \"name\": \"projects/migration-ucaip-training/models/custom_job_TF_20210325211532/versions/custom_job_TF_20210325211532\",\n",
        "      \"deploymentUri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "      \"createTime\": \"2021-03-25T21:42:01Z\",\n",
        "      \"runtimeVersion\": \"2.1\",\n",
        "      \"etag\": \"3vf44xGDtdw=\",\n",
        "      \"framework\": \"TENSORFLOW\",\n",
        "      \"machineType\": \"mls1-c1-m2\",\n",
        "      \"pythonVersion\": \"3.7\"\n",
        "    }\n",
        "  }\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6a9e3ae658cb"
      },
      "outputs": [],
      "source": [
        "# The full unique ID for the model version\n",
        "model_version_name = response[\"metadata\"][\"version\"][\"name\"]\n",
        "\n",
        "print(model_version_name)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9b9af6d5c234"
      },
      "outputs": [],
      "source": [
        "while True:\n",
        "    response = (\n",
        "        cloudml.projects().models().versions().get(name=model_version_name).execute()\n",
        "    )\n",
        "    if response[\"state\"] == \"READY\":\n",
        "        print(\"Model version created.\")\n",
        "        break\n",
        "    time.sleep(60)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sqbcNfhKyIs-"
      },
      "source": [
        "### Prepare input for online prediction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "92334e84c003"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "import json\n",
        "\n",
        "import cv2\n",
        "import tensorflow as tf\n",
        "\n",
        "(_, _), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()\n",
        "\n",
        "test_image_1, test_label_1 = x_test[0], y_test[0]\n",
        "test_image_2, test_label_2 = x_test[1], y_test[1]\n",
        "\n",
        "cv2.imwrite(\"tmp1.jpg\", (test_image_1 * 255).astype(np.uint8))\n",
        "cv2.imwrite(\"tmp2.jpg\", (test_image_2 * 255).astype(np.uint8))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OoqARJbMyIs-"
      },
      "source": [
        "### [projects.predict](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects/predict)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ly6XCNkbyIs-"
      },
      "source": [
        "#### Request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b54YfVXvyIs-"
      },
      "outputs": [],
      "source": [
        "instances_list = []\n",
        "for img in [\"tmp1.jpg\", \"tmp2.jpg\"]:\n",
        "    bytes = tf.io.read_file(img)\n",
        "    b64str = base64.b64encode(bytes.numpy()).decode(\"utf-8\")\n",
        "    instances_list.append({\"key\": img, input_name: {\"b64\": b64str}})\n",
        "\n",
        "request = cloudml.projects().predict(name=model_version_name)\n",
        "request.body = json.loads(json.dumps({\"instances\": instances_list}, indent=2))\n",
        "\n",
        "print(json.dumps(json.loads(request.to_json()), indent=2))\n",
        "\n",
        "request = cloudml.projects().predict(\n",
        "    name=model_version_name, body={\"instances\": instances_list}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"uri\": \"https://ml.googleapis.com/v1/projects/migration-ucaip-training/models/custom_job_TF_20210325211532/versions/custom_job_TF_20210325211532:predict?alt=json\",\n",
        "  \"method\": \"POST\",\n",
        "  \"body\": {\n",
        "    \"instances\": [\n",
        "      {\n",
        "        \"key\": \"tmp1.jpg\",\n",
        "        \"bytes_inputs\": {\n",
        "          \"b64\": \"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAgACADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6E1zw/qemaZY669mkdtqsTPZTpMH85Y3KMcKeOR36444NZGj2/ibWPHaaPeHSLXRbq3jSw1O7u3V3u9zb0ZAh+QIFO4EkliCBjnwv9lfxtrviTxBbW974le/0nQ/h5ohms7m4b92bhVlkEfPIDuwJ6gyADgCuWh1fxP8As6/tGad8H5PiRrHjW6tNd1O/iXUr5Z7mx0uSZlinHODiRQCqrgGTGPmwPyqfClGlnM6Em3TSi/N3Wtnto015H6y+MK08kp14QSqScle6tFxel0+6aZ9d6/rvhXwH4407wWtq+uSXth9pa5jcwKUBIbyxzkL0Ock8nHQV2x0NtN0Gw8a6PDOunXc3liO5GZIGxwG6YBxx1x0zkV4L8Xfij4k8X/Gr4V+HdJtDpdgui3GoajJBAXlkuGvNoUEDcD5MYyuN3zEnpX0B4Q+Iunafdap8OPFCG/sL+PzLkGNgbQB1O7Jxh1JOCOvHXNfUYrh/LPqMo0oKDgvdl10117nzGD4izR5hGdWcp8zs4+umisflx8DNXi/Z/wDHviPTfiP4g+x2WieFtV03U5r9miLw2ilonTIySWijZCB6Yr2X4R/tQT/tC/s56f8AGn4C/AvxTrXiq7jksW1G78NxRlNiRxIrzO5EwiVHAePAfeoO1lIrqv2pf2Xz+1t+z3feC9E1GLSvE2paQtraa1cISXiEqu9tKVydrbMZ5Kkg8jIr234a/Bq7+EngjQPAng3wzB/ZOl6ZFa2tpp/yeWiqFB2Hq2ASeuTz15r9ixHBa+vSp1JXpxXuy6vyfpbXuz8jocUyWCVSirTb1j09V95e+E3hnwXr8dn8QPjLaSWZBguP+EcudKSW6gnSMfLHOrcQh2djCSAxY5BxkzfEDx1H4n8ZyvpEC2WnMAwighMe8hvl3gZyQCB15K5xWNq3iKbVNVk8MW91NZzxLllkt9jL2z0I/DrXCeG47T4seNL3wN4c1nULKPTY2GoX8YYNcSkfKisxwis2ASMnk9AK7f8AiHuQ47CulWlKzfM7S5W+vRfgZQ47zvA4qNako3irK8eZLpfVn//Z\"\n",
        "        }\n",
        "      },\n",
        "      {\n",
        "        \"key\": \"tmp2.jpg\",\n",
        "        \"bytes_inputs\": {\n",
        "          \"b64\": \"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAgACADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD8V5UubRQlxlSvDAtyD6dadbW91fK8lrFI6o6KzrnALHCj8cH8jX3J+1V+wR8adOjsrDxR8EPhzohsg13qKfD+zddRWBF2u0sR42AnIzjJAAzzXmnwh/Yk+D3jX4Q6h478cftgaX4Al/tR4f8AhHdf0eRruVI+IpdkbFiWLsAqgnrXZLBVFWcI6/gc0MVSlSU2eZaX+zdr954Nv/EEt7FNeWyrJHZ2moRn93tYsTuwcg7OBz19q8sa7AUEMf8AvqvoHwX+yz8Vb74gXtn4M+Euq/EbSYpV+y6vf2txptrMOAz+XIysR0xu9M4qf9pn9mf4jJoNprJ+BGgeCn0mHZfQ2OqRl793fAZUDkkAbcd8k1pUw1OUE6e/bf8AEVOs1JqT3P19/aT/AOCMf7RH7Qfx5134zeNf2z7S18Q+PkSWWDSb6406BrSMFYrWNCCAsakDbnOSSeTXg+sf8G3viHwt49ez1jxdY6zqds1veTwT+MzBdqJWnWCYb0DhXe3n2sOGMD4J2HH7IfD3xnc/EPwl4Y8R6t458M28y+EL1NRh1nS3vGXV3a1+w3S4mjCwxxpdCaFSjTNLGRImwk+A6f8AAL9oH4gaX4+tf+Ckn7Vfw4+I2k3fiW6m+HOneFNPn0WDw9piTLLbuUiYGWZsCNYp/tMtqiSbL+b7RMrqvWxVDKamZ89BOg03Q9+deupOpBRotU1CM4OMak/aSUIxkouTbUjmllc0qic60XrGNldX/dtNr/n2+aS5r3XI3ytKz+Jof+CN2r6LYHU/ibqOo2iQzFmmn8eXLfugMbDhwMcdeprg/iV+zX+zx8O9Mu9f8NaRplw9oSr6g0sl0BgdBNMzZ+i9K+svi9P+yv8ADAnRfhl4MfxNdhSDe63fzS2sJHdYpHbfjtu/KvhL9ub4tarruhy2JvJMsdjJFGFj28gKqrgKo9B6VhlvEGMzfDxm8M6N+kpRlJeT5dE/mwoZDiMO+evVb8j/2Q==\"\n",
        "        }\n",
        "      }\n",
        "    ]\n",
        "  },\n",
        "  \"headers\": {\n",
        "    \"accept\": \"application/json\",\n",
        "    \"accept-encoding\": \"gzip, deflate\",\n",
        "    \"user-agent\": \"(gzip)\",\n",
        "    \"x-goog-api-client\": \"gdcl/1.12.8 gl-python/3.7.8\"\n",
        "  },\n",
        "  \"methodId\": \"ml.projects.predict\",\n",
        "  \"resumable\": null,\n",
        "  \"response_callbacks\": [],\n",
        "  \"_in_error_state\": false,\n",
        "  \"body_size\": 0,\n",
        "  \"resumable_uri\": null,\n",
        "  \"resumable_progress\": 0\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jjeBHVKbyIs-"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k5B1LQmQyIs_"
      },
      "outputs": [],
      "source": [
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0NX1ETnIyIs_"
      },
      "source": [
        "#### Response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xXBokAMLyIs_"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"predictions\": [\n",
        "    {\n",
        "      \"key\": \"tmp1.jpg\",\n",
        "      \"prediction\": [\n",
        "        0.033321816474199295,\n",
        "        0.052459586411714554,\n",
        "        0.1548144668340683,\n",
        "        0.11401788890361786,\n",
        "        0.17382356524467468,\n",
        "        0.09015275537967682,\n",
        "        0.19865882396697998,\n",
        "        0.10446509718894958,\n",
        "        0.02987445704638958,\n",
        "        0.048411525785923004\n",
        "      ]\n",
        "    },\n",
        "    {\n",
        "      \"key\": \"tmp2.jpg\",\n",
        "      \"prediction\": [\n",
        "        0.03346974775195122,\n",
        "        0.052550218999385834,\n",
        "        0.15449965000152588,\n",
        "        0.11388237029314041,\n",
        "        0.17408263683319092,\n",
        "        0.08989296108484268,\n",
        "        0.19814379513263702,\n",
        "        0.10520866513252258,\n",
        "        0.02989153563976288,\n",
        "        0.04837837815284729\n",
        "      ]\n",
        "    }\n",
        "  ]\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4aeaf9140e29"
      },
      "source": [
        "### [projects.models.versions.delete](https://cloud.google.com/ai-platform/prediction/docs/reference/rest/v1/projects.models.versions/delete)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e7b24444ac4d"
      },
      "source": [
        "#### Call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8215bb333b22"
      },
      "outputs": [],
      "source": [
        "request = cloudml.projects().models().versions().delete(name=model_version_name)\n",
        "\n",
        "response = request.execute()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "response:migration"
      },
      "source": [
        "#### Response\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "print:migration,new,request"
      },
      "outputs": [],
      "source": [
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_undeploymodel:migration,new,response"
      },
      "source": [
        "*Example output*:\n",
        "```\n",
        "{\n",
        "  \"name\": \"projects/migration-ucaip-training/operations/delete_custom_job_TF_20210325211532_custom_job_TF_20210325211532-1616708584436\",\n",
        "  \"metadata\": {\n",
        "    \"@type\": \"type.googleapis.com/google.cloud.ml.v1.OperationMetadata\",\n",
        "    \"createTime\": \"2021-03-25T21:43:04Z\",\n",
        "    \"operationType\": \"DELETE_VERSION\",\n",
        "    \"modelName\": \"projects/migration-ucaip-training/models/custom_job_TF_20210325211532\",\n",
        "    \"version\": {\n",
        "      \"name\": \"projects/migration-ucaip-training/models/custom_job_TF_20210325211532/versions/custom_job_TF_20210325211532\",\n",
        "      \"deploymentUri\": \"gs://migration-ucaip-trainingaip-20210325211532/custom_job_TF_20210325211532\",\n",
        "      \"createTime\": \"2021-03-25T21:42:01Z\",\n",
        "      \"runtimeVersion\": \"2.1\",\n",
        "      \"state\": \"READY\",\n",
        "      \"etag\": \"Nu2QJaCl6vw=\",\n",
        "      \"framework\": \"TENSORFLOW\",\n",
        "      \"machineType\": \"mls1-c1-m2\",\n",
        "      \"pythonVersion\": \"3.7\"\n",
        "    }\n",
        "  }\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQ-VVaSxJjkd"
      },
      "source": [
        "# Cleanup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rbFGEzYPyIs_"
      },
      "outputs": [],
      "source": [
        "delete_model = True\n",
        "delete_bucket = True\n",
        "\n",
        "\n",
        "# Delete the model using the Vertex AI fully qualified identifier for the model\n",
        "try:\n",
        "    if delete_model:\n",
        "        cloudml.projects().models().delete(name=model_id)\n",
        "except Exception as e:\n",
        "    print(e)\n",
        "\n",
        "if delete_bucket and \"BUCKET_NAME\" in globals():\n",
        "    ! gsutil rm -r gs://$BUCKET_NAME"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "UJ2,12 legacy Custom Training Prebuilt Container TF Keras.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
